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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address -- 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )^ Responsive to communication(s) filed on 12 October 2000 . 
2a)D This action is FINAL. 2b)S This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1 935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) ^ Claim(s) 11-60 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 11-60 is/are rejected. 

7) Q Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)^ The drawing(s) filed on 12 October 2000 is/are: a)S accepted or b)Q objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
1 !)□ The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

1 3) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (f). 

a)QAII b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2.Q Certified copies of the priority documents have been received in Application No. . 



3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) Q Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 

Attachment(s) 



1 ) £3 Notice of References Cited (PTO-892) 

2) [Zl Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) ^ Information Disclosure Statement(s) (PTO-1449) Paper No(s) 2.3.5 . 



4) CD Interview Summary (PTO-413) Paper No(s). 

5) CD Notice of Informal Patent Application (PTO-1 52) 

6) D Other: 
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PTO-326 (Rev. 04-01) 



Office Action Summary 



Part of Paper No. 6 





' Application/Control Number: 09/689,565 
Art Unit: 2625 



Page 2 



DETAILED ACTION 



Claim Rejections - 35 USC § 103 



1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 1 1-22 and 27-60, are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Maurer et al (U.S. patent 6,272,231) in view of Singh (U.S. patent 6,204,860). 

Regarding claim 11, Maurer et al discloses a wavelet-based facial motion capture for 
avatar animation comprising: 

Determining the calibration parameter of a camera (column 11, lines 63-67, camera 
parameter are computed through a calibration process). 

Marking salient features of an object with markers for motion tracking (column 5, lines 
29-46, a collection of facial images is marked with node location at defined position of head, this 
matching process is called elastic bunch graph matching such as persons of different ages or 
races). 

Acquiring a plurality of initial 2-D images of the object (column 6, line 66 through 
column 7, line 14, refer to 2D vector). 
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Calculating 3-D locations of the salient features of the object in accordance with the 
calibration parameter of the camera; receiving a chronologically ordered sequence of 2-D images 
of the object (column 10, lines 51-67, a 3D bunch graph is created which is analogous to the 2D 
approach. Projections of the 3D graph are then used in the matching process). 

However Maurer et al fails to disclose "salient On the other hand Singh in the same 
field of tracking objects teaches column 13, line 59 through column 14, line 4, it allows the 
animator intuitive control over many salient visual features of wrinkle formation and 
propagation. 

Therefore it would have been obvious to one having ordinary skill in the art at the time of 
the invention was made, to modify Maurer et al invention according to the teaching of Singh 
because it provides a coarse-scale representation of the object surface and a primitive that can be 
directly manipulated while highlighting and tracking the deformable features of the object to 
achieves accuracy and image quality. 

Regarding claim 12, Maurer et al discloses the method of, further comprising the steps of: 
locking onto the salient features; and detecting loss of lock and hence the need for re-locking 
onto the salient features (column 9, lines 7-20, detecting certain geometrical constrains and a 
model containing the precise shape information of the tracked face). 

Regarding claim 13, Maurer et al discloses the method of wherein the step of tracking 
comprises the steps of: determining a surface normal for each salient features; tracking the 3-D 
global motion f the face in each image; and tracking the 3-D local motion of t e face in each 
image (column 12, lines 3-6 and lines 24-27, generating of 3-D model). 
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Regarding claim 14, Maurer et al discloses the method of comprising the further step of 
repeating the locking and tracking steps after the detecting step (column 7, lines 7-21, process is 
repeat for all Gauss levels until the finest resolution level is reached). 

Regarding claim 15, Maurer et al discloses the method, wherein the step of selecting 
comprises recognizing salient facial features and the step of tracking comprises tracking the 
motion of the salient facial features (column 5, lines 29-46, a collection of facial images is 
marked with node location at defined position of head, this matching process is called elastic 
bunch graph matching such as persons of different ages or races. 

Regarding claim 16, Maurer et al discloses the method, wherein the step of selecting 
comprises fixing markers to the face and the step of tracking comprises tracking the motion of 
the markers (column 3, lines 12-29, provided the facial sensing process which locates the 
person's face and corresponding facial features). 

Regarding claim 1 7, Maurer et al discloses the method, wherein a first set of markers 
identifies global motion and a second set of markers identifies local motion of the face (column 
7, lines 42-51, refer to second computation). 

Regarding claim 1 8, Maurer et al discloses the method wherein the markers comprise at 
least two color (Fig. 17, column 14, lines 8-21, dynamic texture generation refer to color). 

Regarding claim 19, Maurer et al discloses the method, wherein the two colors are 
contrasting (column 14, lines 9-25, Fig. 17, refer to black and white). 

Regarding claim 23, Maurer et al discloses the method, wherein the step of selecting 
comprises wearing a head-set with markers (column 12, lines 3-7, where markers are used). 
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Regarding claim 27, Maurer et al discloses the method, wherein the acquired 2-D images 
include at least two views of the face with markers in a neutral state at different orientations 
(column 10, lines 31-50, refer to two poses and different graphs). 

Regarding claim 28, Maurer et al discloses the method, wherein the two views are 
orthogonal (Fig. 6, item 62, column 4, lines 61-67 refer to orthogonal). 

Regarding claim 29, Maurer et al discloses the method, wherein the acquired 2-D images 
comprise front, forehead chin, angled-right, angled-right-tilted-up, angled-right-tilted-down, 
angled-left, angled-left-tilted-up, angled-left-tilted-down, full-right-profile, full-right-profile 
tilted-up, full-right-profile-tilted-down, full-left-profile, full-left-pro file-tilted-up, and full-left- 
profile-tilted-down views of the face with markers in the neutral state (column 3, line 54 through 
column 4, line 8, the facial feature of the model graph in the upper left hand and left to right used 
to locate a similarity and column 12, lines 28-34, refer to eyes, nose, mouth, and chin). 

Regarding claim 30, Maurer et al discloses the method, wherein the acquired 2-D images 
comprise front, forehead, chin, full-right-profile, and full-left-profile views of the face with 
markers in the neutral state (column 12, lines 28-34, facial contour). 

Regarding claim 31, Maurer et al discloses the method of claim 1 wherein he acquired 2- 
D images include a plurality of views of the face with markers in at lea one action state (column 
6, line 66 through column 7, line 14, refer to 2D vector). 

Regarding claim 32, Maurer et al discloses the method, wherein the action states of the 
face comprise smiling lips, kissing lips, yawning lips, raised eyebrows, and squeezed eyebrows 
(column 5, lines 9-27, refer to eyebrow and lip). 
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Regarding claim 33, Maurer et al discloses the method, wherein the acquired 2-D images 
of the face in an action state include at least two views at different orientations (column 5, lines 
30-46, different facial image). 

Regarding claim 34, Maurer et al discloses the method, wherein the two views are front 
and angled-right (column 3, line 54 through column 4, line 8, the facial feature of the model 
graph in the upper left hand and left to right used to locate a similarity and column 12, lines 28- 
34, refer to eyes, nose, mouth, and chin). 

Regarding claim 35, Maurer et al discloses the method, wherein the step of selecting 
comprises fixing markers to the face and the step of calculating comprises calculating the 3-D 
locations of the markers placed on the face (column 13, lines 9-18, 3-D morphing and geometric 
transformation). 

Regarding claim 38, Maurer et al discloses the method, wherein the step of calculating the 
3-D locations of the local markers in each action state comprises the steps of estimating the 
orientation and position of the face in each 2-D image of the action state to conform to the 3-D 
and 2-D locations of the global markers under a perspective projection model; and calculating 
the 3-D locations of the local markers to conform to the estimated orientation and position of the 
face and the 2-D locations of the local markers under a perspective projection model (column 6, 
line 66 through column 7, line 14, refer to 2D vector and column 10, lines 51-67, a 3D bunch 
graph is created which is analogous to the 2D approach. Projections of the 3D graph are then 
used in the matching process). 
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Regarding claim 41, Maurer et al discloses the method, wherein detecting the global 
markers comprises: determining visibility indices of global markers designing correlation filters 
for the global markers detecting the global markers by applying elliptical correlation filters in a 
neighborhood of the global markers; and eliminating superfluous and multiple detected locations 
(column 8, lines 23-41, refer to generating the filter). 

Regarding claim 42, Maurer et al discloses the method, wherein the step of estimating 
comprises calculating the position and orientation of the face to conform to the 3-D locations and 
the detected locations of the global markers under a perspective projection model (column 7, 
lines 1-20, comparing and calculation). 

Regarding claim 49, Maurer et al discloses the method, wherein the step of calculating an 
action vector comprises the steps of calculating the difference between the 2-D locations of the 
local markers detected in an image and the 2-D locations of the same markers corresponding to 
the neutral face modifying the difference to conform to the orthographic projection calculating 
the 3-D displacements of the local markers with respect to their location in the neutral face; and 
calculating the amount of facial actions conforming to the 3-D displacements of the local 
markers (column 6, line 66 through column 7, line 14, refer to 2D vector and column 10, lines 
51-67, a 3D bunch graph is created which is analogous to the 2D approach. Projections of the 3D 
graph are then used in the matching process). 

Regarding claim 50, Maurer et al discloses the method, wherein the step of calculating an 
action vector comprises the steps of calculating the 2-D locations of the local markers 
corresponding to the neutral face using the global motion found for the current image; 
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calculating the 2-D locations of the local markers corresponding to the action faces using the 
global motion found for the current image; calculating the distance between the detected 
locations, the distance between the neutral locations, and the distance between the action 
locations of the markers at the right and left corners of the lips (Fig. 6, column 5, lines 5-35, refer 
to different location of face). 

Regarding claim 57, Maurer et al discloses the method, wherein the step of estimating 
local motion comprise calculating a vector of weights representing fractions of maximum actions 
of the object conforming to the 3-D locations of local markers in the neutral and action states and 
the detected 2-D locations of the local markers under a perspective projection model (column 1, 
lines 43-56, refer to maximum similarity and location of model). 

Regarding claims 20-22, recite similar limitation as claims 18 and 19 and are similarly 
analyzed. 

Regarding claims 36-37 and 39, recite similar limitation as claims 1 1, 35 and 38 are 
similarly analyzed. 

Regarding claims 40 and 43-45, recite similar limitation as claims 29 and 38 are similarly 
analyzed. 

Regarding claims 46-48 and 51-56, recite similar limitation as claims 1 1 and 35are 
similarly analyzed. 

Regarding claims 58-60, recite similar limitation as claim 1 lis similarly analyzed. 
3. Claims 23-26, are rejected under 35 U.S.C. 103(a) as being unpatentable over 
as applied to claims above, and further in view of Ferre et al (U.S. 6,175,756). 
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Regarding claim 23 , Maurer et al and Singh are silent about "headset". On the other hand 
Ferre et al teaches (column 14, lines 27-28, comprising markers coupled to the headset for 
providing reference points on computer image. 

Therefore it would have been obvious to a person of ordinary skill in the art at time the 
invention was made, to modify Maurer and Singh invention according to the teachings of Ferre 
et al because it provides plurality of prerecorded images of the first coordinate system, and 
identifies a desired and improve prerecorded image associated with the position of the remote 
unit. 

Regarding claims 24-26, recite similar limitation as claim 23 and is similarly analyzed. 

Other prior art cited 

4. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

U.S. patent (6,052,132) to Christian et al is cited for technique for providing a computer 
generated face having coordinated eye and head movment. 
keyboard with flexible display and prompt capability. 

U.S. patent (5,805,745) to Graf is cited for method for locating a subject's lips in a facial 

image. 

U.S. patent (6,009,210) to Kang is cited for hand-free interface to a virtual reality 
environment using head tracking. 

U.S. patent (6,016,148) to Kang et al is cited for automated mapping of facial images to 
animation wireframes topologies. 

U.S. patent (6,028,960) to Graf et al is cited for face feature analysis for automatic 
lipreading and character animation. 
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Contact Information 

5. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Seyed Azarian whose telephone number is (703) 306-5907. 
The examiner can normally be reached on Monday through Thursday from 6:00 a.m. to 7:30 
p.m. If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Bhavesh Mehta, can be reached at (703) 308-5246. 

Any response to this action should be mailed to: 

Assistant Commissioner for Patents 
Washington, D.C. 20231 
Or faxed to: 

(703) 872-9314, ("draft" or "informal" communications should be clearly labeled 
to expedite delivery to examiner). 

Hand delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, 
Arlington, VA., Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application should be directed to 
T.C. customer service office whose telephone number is (703) 306-0377. 



Seyed Azarian 
Patent Examiner 
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